Repository navigation
fix(engine-mcp): a one-word name spelled in prose has to earn its lead (PRD-865 follow-up) - #647
Merged
Merged
Conversation
A follow-up review of the merged PRD-865 change found that a spelled one-word name started at NAME_BASE_LEAD = 2, above the situation scores a prose request produces, so ordinary English that happens to be a symbol took row 1: "camera follows the player with a third person view" answered the React `View` rectangle, "the enemy velocity should push the player back" answered a `three/tsl` velocity buffer, and `scene`, `depth` and `connect` did the same. The base commit answered all of them correctly and the merged change did not. A one-word name spelled inside a longer request now scores (NAME_BASE_LEAD + NAME_SHARE_LEAD) * share and must hold a real share of the request (SINGLE_WORD_SHARE_FLOOR) and clear the name index's weight floor. Names spelled as the whole request, and multi-word names, are unchanged, so `add godrays`, `enable ssgi`, `use rapier`, `GPUParticles3D sparks` and `Area3D trigger` still lead. Held out: recall@1 0.400 -> 0.660 (was 0.620), recall@5 0.580 -> 0.760, MRR 0.701; every A/B gate passes and `caps:recall` stays 0.938053. Also closes the two places that still told an agent to open a detail call for every hit — the discovery-gate hook and the shared ponytail skill — and corrects the alias comment in the manifest spec: those rows are recalled through the name channel and the camelCase situation split, not through summaries.
Conflicts: develop (#632) shipped a stale engine-server.mjs bundle without the PRD-865 name lead; rebuilt it from engine-mcp source. scaffold.spec.ts moved to two-hash pins; re-pinned with pnpm scaffold:pin (all 16 kits move shared only).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #644 (PRD-865), which the merge-queue landing shipped with a defect a third review pass then found. One commit, off
developata6f32923b.The defect
A spelled one-word name started at
NAME_BASE_LEAD = 2, above the situation scores a prose request produces, so ordinary English that happens to be a symbol took row 1 — while the base commit (ecd2a1a56) answered all of these correctly:a6f32923b)View(a React rectangle) 2.26defineGamedefineGame,Viewat 0.62velocity(three/tsl)velocitygonedepth(three/tsl)AtmosphereAtmosphereScene2.95Scenegoneconnect(WebTransport) in the top 3Joint3Dat row 2connectgoneThe fix
A one-word name spelled inside a longer request now scores
(NAME_BASE_LEAD + NAME_SHARE_LEAD) * share, and must holdSINGLE_WORD_SHARE_FLOORof the request and clear the name index'sSYMBOL_WEIGHT_FLOOR. A request that is the name, and multi-word names, are untouched, so the names that should lead still do:add godrays,enable ssgi,use rapier,GPUParticles3D sparks,Area3D trigger,characterbody3d.The four false leads above are now red-green tests in
packages/engine-mcp/__tests__/search.spec.ts(they fail ona6f32923b, pass here), plus one keeping the names that must still lead.Effect on the frozen held-out set
All four A/B gates pass;
pnpm caps:recallstays 0.938053 (exit 0).Also in this commit
agent-files/.claude/hooks/discovery-gate.mjsandagent-files/.agents/skills/ponytail/SKILL.mdstill told an agent to openengine_capability_detailon every hit — the two places the router rule change missed. Both now read every row, and the sixteen scaffold trees are re-measured.capability-manifest.spec.tsnamed the wrong cause:SurfaceCache/RadianceCacheare recalled through the name channel andGIBudget/ShadowFilterBudgetthrough situations after the camelCase split — not through summaries.Gates
pnpm exec vitest run packages/engine-mcp158 passed / 1 skipped;packages/create-threenative/__tests__/scaffold.spec.tsgreen with the re-measured trees;instruction-budget,kit-conventions,capability-manifest,mcp-tool-schemasgreen;pnpm typecheck,pnpm lint,pnpm check:docs,pnpm sync:agents --check,pnpm qualitygreen;pnpm caps:recallexit 0;scripts/capability-search-ab.ts --base ecd2a1a56exit 0.